Open Access 



Research 



BMJ Open Patient and public involvement 
in the early stages of clinical trial 
development: a systematic cohort 
investigation 

Carrol Gamble, 1 Louise Dudley, 1 Alison Allam, 2 Philip Bell, 3 Heather Goodare, 4 
Bee Hanley, 5 Jennifer Preston, 6 Alison Walker, 4 Paula Williamson, 1 Bridget Young 7 



To cite: Gamble C, Dudley L, 

Allam A, etal. Patient and 

public involvement 

in the early stages of clinical 

trial development: a 

systematic cohort 

investigation. BMJ Open 

2014;4:e005234. 

doi:1 0.1 1 36/bmjopen-201 4- 

005234 

► Prepublication history and 
additional material is 
available. To view please visit 
the journal (http://dx.doi.org/ 
10.1136/bmjopen-2014- 
005234). 



Received 11 March 2014 
Accepted 30 June 2014 



CrossMark 



For numbered affiliations see 
end of article. 



Correspondence to 

Professor Carrol Gamble; 
R.Gamble@liverpool.ac.uk 



ABSTRACT 

Background: Randomised controlled trials (RCTs) 
are considered particularly likely to benefit from patient 
and public involvement (PPI). Decisions made by 
professional researchers at the outset may go on 
to have a significant impact on the potential for PPI 
contributions. 

Objective: To increase knowledge of PPI within the 
early development of RCTs by systematically describing 
the reported level, nature and acceptability of proposed 
PPI to the funders. 

Methods: Documentation from the outline 
application process for all RCTs that received funding 
from the Health Technology Assessment (HTA) 
Programme 2006-2010 was requested. For each 
application, data were extracted on trial characteristics, 
references to PPI in the development of the outline 
application and funding Board feedback, and plans for 
PPI in the full application and after the trial was 
funded. 

Results: 110 applications were eligible with outline 
applications available for 90 (82%). The cohort covered 
a wide range of interventions and conditions. 54% (49/ 
90) provided some information about PPI. 26 (28.9%) 
indicated PPI within the development of the outline 
application itself; 32 (35.6%) planned involvement in 
the full application and 43 (48%) once the trial was 
funded. Recruitment at diagnosis and surgical 
interventions were less likely to describe PPI. Blinded 
trials and trials in which participants may receive 
placebo only, more frequently described PPI activity. 
The HTA commissioning Board feedback rarely referred 
to PPI. 

Conclusions: Incorporation of PPI within the 
development of the outline application or specification 
of plans for future involvement was low. Funder 
requests for applicants to provide information on PPI 
and justification for its absence should be welcomed 
but further research is needed to identify the impact of 
this on its contributions to research. Comments on PPI 
by reviewers should be directional rather than state 
that an increase is required. Challenges facing 
applicants in initiating PPI prior to funding need to be 
addressed. 



Strengths and limitations of this study 



This is the first study to systematically investigate 
patient and public involvement (PPI) early in the 
development of funding applications and across 
a wide range of trials. 

Presents results of a large unselected cohort 

addressing concerns of selective reporting of 

case studies within the literature. 

Is the first study to consider clinical and trial 

characteristics in relation to the model of PPI 

adopted. 

Cohort is limited to a single funder covering all 
randomised controlled trials receiving funding 
between 2006 and 2010. 
Documentation could not be made available for 
unsuccessful applications. However, PPI was not 
mandatory within the period of the cohort, and 
that on its own would not have led to a decision 
that the outline application should not progress. 
As the cohort was identified by receipt of 
funding during the period rather than the year 
that the outline application was submitted for 
consideration, fewer trials were available in the 
extremes of the cohort. 

Changes to funder requirements for PPI have 
changed and this may impact generalisability. 



BACKGROUND 

Public involvement in research is described 
as research being carried out 'with' or 'by' 
members of the public rather than 'to', 
'about' or 'for' them (http://www.invo.org. 
uk/ find-out-more/ what-is-public-involvement- 
in-research-2/ last accessed 5 September 
2013). It has been suggested that clinical 
trials are particularly likely to benefit from 
patient and public involvement (PPI). 1 2 PPI 
can start at various stages of a trial and influ- 
ence many aspects. Increased recognition 
that patients and public are stakeholders in 
research has led to increasing calls that they 
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be represented within that research process. The UK 
Department of Health (2006) Best Research for Best Health 
guidelines^ state that PPI must be included in all stages of 
the research process including priority setting, defining 
research outcomes, selecting research methodology, 
patient recruitment, interpretation of findings and dis- 
semination of results. Many funding bodies now request 
researchers to provide evidence of PPI when submitting 
their proposals. 4 The National Institute of Health 
Research (NIHR) Health Technology Assessment (HTA) 
programme encourages applicants to consider whether 
the 'scientific quality, feasibility or practicality of the pro- 
posal can be improved by patient and public involve- 
ment' (http://www.hta.ac.uk/ public/) . 

Staniszewska et af discussed the importance of PPI 
being implemented at the design stage of studies to opti- 
mise the impact and relevance of research. During the 
design stages of a trial many decisions are made that 
determine the relevance and conduct of the proposed 
research: the precise specification of the research ques- 
tion, including the outcomes to be measured; visit sche- 
dules; methods of data collection; recruitment and 
consent procedures. Fudge et af suggest that decisions 
made by professional researchers at the outset of a study 
have a cumulative and significant influence on the 
potential for PPI to impact on a study and that involve- 
ment is more difficult to achieve once studies are under 
way. It is therefore important to consider PPI activity in 
the early stages of the research process. However, little is 
known about how or when researchers incorporate PPI 
or what impact may stem from that involvement. 
Concerns have been expressed that the existing litera- 
ture is selectively reported with many reports aiming to 
'make the case' or 'convince the sceptics' about PPI. 7 
Furthermore, as much reporting has involved single case 
studies, generalisability of the PPI literature is limited. 7 

The aim of this paper is to increase the knowledge of 
PPI within the early stage design and development of 
clinical trials by systematically studying an unselected 
cohort of outline applications that were in receipt of 
funding from the HTA programme between 2006 and 
2010. Specific objectives are to identify whether and how 
PPI is described within the early development of a grant 
application for funding; examine how PPI contributions 
within the development of the outline application were 
reviewed by the HTA Board; describe applicants' PPI 
plans should their application be successful; and 
describe variations in PPI in relation to time of funding 
and trial characteristics. 



METHODS 

We examined a cohort of randomised controlled trials 
(RCTs) that received funding from the NIHR HTA) pro- 
gramme in the UK between 2006 and 2010. During the 
period of the cohort the HTA programme encouraged 
PPI, and allowed for resources to support PPI activity to 
be requested; however, PPI was not mandatory. Inclusion 



of a PPI contributor on the Trial Steering Committee 
(TSC) was recommended. The HTA programme has a 
two-stage application process, comprising an outline 
application followed by a full application for those suc- 
cessful at the outline stage (http://www.hta.ac.uk/ 
funding/clinicaltrials/index.shtml, accessed 30 April 
2012). We requested all documentation relevant to the 
outline applications from the HTA. This comprised the 
commissioning briefs (if applicable) in which the topic 
of research driven by funders rather than applicants, 
outline applications and the minutes of the relevant 
Board meetings at which the outline applications were 
considered and which contained feedback for the appli- 
cant. Prior to the release of these documents the NIHR 
HTA contacted the chief investigators of the trials 
involved informing them that their names were released 
as these were readily available within the public domain. 
We signed a confidentiality agreement and the NIHR 
HTA redacted sensitive information regarding budget 
information and names of co-applicants (not available 
within the public domain) before releasing the docu- 
mentation to us. 

Once the trial was funded we established a PPI advis- 
ory group of five members with experience of providing 
PPI in RCTs. Members were recruited through an open 
and transparent process. Advertisements for members 
were placed on various PPI websites and applicants who 
expressed an interest were sent an application pack 
which included a detailed remit and role description. 
Applicants were then short listed and invited to take 
part in an informal interview over the phone, and had 
the opportunity to ask further questions about the role 
of the group. To assist in extracting and analysing the 
information on PPI within the outline applications an 
access database was developed by CG. The advisory 
group commented on the data extraction tool and made 
recommendations for changes to variable descriptors 
and coding categories. All suggestions were discussed in 
a face-to-face meeting and agreement reached on their 
incorporation within the project. 

Data were extracted to characterise the cohort and to 
describe PPI activity within the application process and 
plans for involvement once trial funding was secured. 
Trial characteristics linked to trial complexity or thought 
to be barriers or facilitators to recruitment 8 were also 
extracted. In brief, the extracted data included 
characteristics of the trial design and setting, proposed 
start date, disease or condition under study, type of inter- 
vention, participant characteristics, recruitment setting 
and any text that described or was relevant to PPI. Data 
were extracted by three reviewers (LD, JP and CG) . The 
extracted text was anonymised by replacing any identify- 
ing details with a general term in brackets [term], or 
using [...] to indicate removed text (see online supple- 
mentary table SI). 

Text extracts were examined to determine whether 
PPI had been described and if so whether PPI had been 
undertaken during the development of the outline 
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application (OL), was planned to occur during the 
development of the full application (F) and/or was 
planned to occur after a trial was funded (T). The text 
was also examined to determine the role of the PPI con- 
tributor's input. This was categorised as: managerial 
(M); responsive (R) and membership of Trial Steering 
Committee (TSC). We categorised PPI contributors as 
managerial if they were described as co-applicants or 
involved in the management of the trial. We categorised 
PPI contributors as having a responsive role if descrip- 
tions of their input were largely confined or targeted 
towards a particular aspects of the application or trial, or 
if PPI contributions were on an 'as required' basis. 
Descriptions of PPI in the outline applications were 
often limited, and we were therefore required to devise 
coding 'rules' (based on our knowledge of grant applica- 
tion development and clinical trial implementation pro- 
cesses) in order to categorise the descriptions. The 
codes were developed by CG and LD after reading the 
PPI descriptions and then reviewed by JP. CG and LD 
independently categorised the PPI descriptions. 
Disagreements between CG and LD were discussed and 
agreement reached. All categorisations were cross 
checked by JP. The coding rules (box 1) and the classifi- 
cations (see online supplementary table SI) were 
reviewed by the PPI advisory group and no changes were 
requested. 

The cohort was identified as RCTs that actively 
received funding from the NIHR HTA programme 
between 2006 and 2010. In considering trends over 
time, the year the outline application was submitted was 
used. 



Box 1 Coding rules used to inform the categorisation of 
patient and public involvement (PPI) descriptions in outline 
applications 



Coding rules for categorising stage of involvement 

1. If a PPI contributor is described as a member of the research 
team or lay member' categorise them as inputting across all 
future stages of the study. 

2. The design of the study is determined within the full applica- 
tion stage so if a PPI contributor is described as inputting 
into the design of the study categorise their input as starting 
no later than at development of the full application. 

3. If a PPI contributor's role is confined to Trial Steering 
Committee membership (which is usually agreed by the 
funders and follows the funding decision) categorise their 
input as starting after the full application regardless of the 
tense of the sentences describing their involvement. 

Coding rules for categorising role 

4 If a PPI contributor's role is described as managerial or as a 
co-applicant or referred to as a part of the team categorise 
their role as managerial. 

5 If a PPI contributor's role is confined to a panel or an advisory 
group categorise their contribution as responsive. 

6 If a PPI contributor's role is limited to a specific aspect of the 
trial then categorise their input responsive. 



Specific conditions were selected to consider in more 
detail. These were identified either by their strong 
history of PPI or the establishment of an NIHR Clinical 
Research Networks for a specific condition (http://www. 
nihr.ac.uk/ infrastructure/Pages/ infrastructure_clinical_ 
research_networks.aspx, last accessed 5 September 
2013). The categories were mental health, pregnancy 
and childbirth, HIV/ AIDS, cancer, stroke, paediatrics, 
diabetes and dementias and neurodegenerative diseases. 

Categorical data were summarised using descriptive 
statistics with numbers and percentages. % 2 Tests or 
Fishers exact tests were used as appropriate. 

Ethical approval was obtained from the University of 
Liverpool Institutional Ethics Board and the study was 
funded by INVOLVE and managed by NIHR HS&DR. 

RESULTS 

Between 2006 and 2010, 111 RCTs received funding 
from the NIHR HTA Programme. Of these 110 were 
required to submit an outline application and were eli- 
gible for inclusion within the cohort. The dates of sub- 
mission of the associated outline applications were 
between 2003 and 2008. Copies of the outline applica- 
tions were available for 90 (82%) and comments from 
the Board on the outline application were available for 
77 (70%). There were five trials in which there was a dis- 
agreement between data extractors, each of which 
related to the presence/absence of PPI text and each 
were resolved by referral to the application form. 

Table 1 provides a summary of the conditions and 
interventions. Trials were funded across a wide range of 
clinical conditions, most of which were for long-term 
conditions, with the most common area of study being 
trials in mental health (17%). The majority of trials 
(82.2%) were aimed at treatment of a condition with 
14.4% being prevention trials. The trials used a wide 
range of interventions. Over a third investigated a medi- 
cinal product and just under a fifth each evaluated 
behavioural interventions (18.9%), surgical techniques 
(17.8%) and devices (16.7%). 

Table 2 describes the characteristics of trial partici- 
pants and features of the trial designs. Over three- 
quarters of the trials recruited adults only, with paediat- 
ric trials accounting for 16.7% of the cohort. The 
majority of the trials were not gender-specific and 
approximately a quarter recruited participants at the 
time of diagnosis. Trial recruitment was most commonly 
conducted within secondary care (61%). Just under a 
quarter of the trials involved blinding of the treating 
clinical team or the participants. Just over 15% of trials 
used a placebo with all participants receiving an active 
intervention in one-third of these, indicating the use of 
a double-dummy design. For example, in a double- 
dummy trial comparing two treatments A and B trial 
participants would receive either treatment A and a 
placebo of treatment B, or a placebo of treatments A 
and B. 
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Table 1 Summary of conditions and interventions 




N=90 

Number of 
trials 

n (%) 



Categories not mutually exclusive. 

fTopic of research identified by the funders for which collaborative 
applications are invited as opposed to an open call in which the 
applicants suggest the topic of research. 



Of the 90 outline applications for which documenta- 
tion was available 49 (49/90=54%) provided some infor- 
mation on PPL Extracts from each application along 
with the classifications on the stage and role of PPI are 
provided in online supplementary table SI. Table 3 sum- 
marises the classifications across the trials. The first row 
of table 3 shows that there were 19 applications in which 



Table 2 Trial participant and design characteristics 




Trial participant and design 

characteristics (N=90) n (%) 

Age group 

Adults only 69 (76.7) 



Paediatrics only 15 (1 6.7) 

Adults and paediatrics 5 (5.6) 

Unclear 1 (1) 
Gender 

Female 6 (6.7) 

Male 0 (0) 

Male and female 84 (93.3) 

Recruiting newly diagnosed patients 24 (26.7) 

Trial recruitment setting* 

Secondary 68(61) 

Primary 53 (58.9) 

Community 12(13.3) 

Emergency 8 (8.9) 

Tertiary 6 (6.7) 

Social care 6 (6.7) 

Blinded trialt 43 (47.8) 

Clinician blind 15/43 (34.9) 

Participant blind 1 9/43 (44.2) 

Trial involves a placebo 14 (15.6) 
If a placebo is involved, do all participants 5/14 (35.7) 
get an active intervention? 

'Categories not mutually exclusive. 



tTwenty -three trials in which blinding only related to the outcome 
assessor. 



the text provided within the outline described PPI 
occurring at all three stages (within the outline applica- 
tion, full application and once the trial was funded). Of 
these 19 applications the role was managerial (M) at 
each stage in 13. In the remaining six there was variation 
in the roles across the stages, or it was unclear when a 
statement indicated that PPI would occur but no detail 
was provided to allow classification. An example of a 
description that we were unable to categorise is: 

Investigators have, and will continue to, collaborate with 
service users. 

Twenty-six applications (26/90=28.9%) specified a role 
of PPI within the development of the outline applica- 
tion. In 13 this was managerial, on a responsive basis in 
seven, unclear in two and 'other' approaches used in 
four (eg, a patient survey or pilot feedback). Within the 
'other' approaches it was difficult to determine conclu- 
sively whether this was PPI or whether they were exam- 
ples of data collection aimed at ascertaining the public 
opinion. In the three applications that specified use of a 
survey the extent of the distribution of the survey was 
unclear in two. PPI was planned to occur within the full 
application for 32 trials (35.6%). In 18 this was manager- 
ial, responsive in 9 and unclear in 5. Forty-three (48%) 
applications indicated that PPI was planned after the 
trial was funded. This was as: managerial in 22; 



Long-term condition 


51 


(56.7) 


Rare condition 


2 


(2.2) 


Conditions expected to reduce lifespan 


27 


(30.0) 


General shortening 


14/27 


(51 .9) 


Rapid mortality 


13/27 


(48.2) 


Conditions under study* 






Mental health 


15 


(16.7) 


Heart disease/condition 


3 


(3.3) 


Haematology/phlebology 


7 


(7.8) 


Infections 


7 


(7.8) 


Musculoskeletal 


6 


(6.7) 


Cancer 


3 


(3.3) 


Renal 


4 


(4 4) 


Childbirth and pregnancy 


2 


(2.2) 


Obstetrics and gynaecology 


2 


(2.2) 


Addiction 


4 


(4.4) 


Dermatology 


2 


(2 2) 


Gastroenterology 


6 


(6 7) 


Diabetes 


2 


(2 2) 


Obesity/nutrition 


3 


(3.3) 


Falls in the elderly 


3 


(3.3) 


Sleep disorders 


2 


(2.2) 


Stroke 


4 


(4 4) 


Dental 


2 


(2 2) 


Degenerative neurological disorders 


4 


(4.4) 


Respiratory 


4 


(4.4) 


Other 


5 


(5.6) 


Aim of intervention 






Treatment 


74 


(82.2) 


Prevention 


13 


(14.4) 


Diagnostic 


3 


(3.3) 


Nature of interventions used* 






Drug 


31 


(34.4) 


Behavioural 


17 


(18.9) 


Device 


15 


(16.7) 


Surgery 


16 


(17.8) 


Comparison with conservative 


7 


(43.8) 


management (n=16) 






Physical, eg, exercise 


11 


(12.2) 


Educational 


10 


(11.1) 


Community care 


5 


(5.6) 


Other 


4 


(4.4) 


Commissioned brieff 


45 


(50.0) 



4 
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Table 3 Summary of stages of PPI by stage and role 



Stage 



Number of outline 
applications 



Role 



Outline (O) Full (F) Trial (T) N (%) n=90 



OFT 



Number of 

outline 

applications 



Comments 



Y 


Y 


Y 


19 (21.3) 


M 


M 


M 


13 










R 


M 


M 


1 










R 


R 


R 


2 










R 


U 


U 


1 










u 


R 


R 


1 










u 


u 


u 


1 


Y 


Y 


NS 


2 (2 2) 


R 


R 




1 










0 


R 




1 


Y 


NS 


Y 


2 (2 2) 


R 




o 


1 










R 


- 


TSC 


1 


Y 


NS 


NS 


3 (3.4) 


0 






3 


NS 


Y 


Y 


8 (9.0) 




M 


M 


3 












R 

R 


R 
M 


2 
1 












U 


U 


2 


NS 


Y 


NS 


1 (1-1) 




R 




1 


NS 


NS 


Y 


12 (13.5) 






M 


3 














R 


1 














TSC 


7 














0 


1 


NS 


NS 


NS 


40 (44.9) 








40 


NS 


U 


U 


2 (2.2) 




U 


U 


1 












M 


M 


1 



2 had multiple approaches alongside 
M in the main trial (TSC; TSC plus R) 



0=described as informal contacts 
0=scale development 
Consulted in pilot study 
0=clinical studies group; survey; 
Service user Forum 
In 1 also TSC & R in trial 
In 1 TSC too 



2 above also listed TSC alongside 

higher order approach 

Piloted and then refined with users 



Unclear when PPI initiated but when 
it starts it is at M 



M, managerial; NS, none specified; O, other; PPI, patient and public involvement; 
U, unclear; Y, yes. 



R, responsive; TSC, member of Trial Steering Committee; 



responsive in 6; a member of the TSC in 8; unclear in 5 
and other in 2. Table 4 provides a summary of the speci- 
fication of PPI within the outline applications by the 
disease area. The numbers of outline applications by 
year with and without details of PPI are displayed in 
figure 1, with figure 2 showing the percentage of 



applications with PPI. Figure 1 shows a general trend for 
increasing the number of funded applications; however, 
the proportion of those containing PPI fluctuates, 
ranging from approximately one half to two-thirds 
(figure 2), with the exception of 2003, for which only 
one application was available. 



Table 4 PPI specified within the outline application by disease area 





PPI details in outline text 






Disease/condition category 


Yes 


No 


Totals 


p Value 


Pregnancy and childbirth 


1 (50.0) 


1 (50.0) 


2 (2.2) 


0.51* 


Cancer 


1 (33.3) 


2 (66.7) 


3 (3.3) 




Stroke 


3 (75.0) 


1 (25.0) 


4 (4.4) 




Mental health 


8 (61.5) 


5 (38.5) 


13 (14.4) 




Paediatricst 


8 (53.3) 


7 (46.7) 


15 (16.7) 




Degenerative neurological disorders 


4 (100.0) 


0 (0.0) 


4 (4.4) 




Other 


24 (49.0) 


25 (51.0) 


49 (54.4) 




Total 


49 (54.4) 


41 (45.6) 


90 (100.0) 





*Fisher's exact test. 

tPaediatrics cuts across specific topics. For example, a trial may be in a mental health disorder affecting young people. All such trials were 
classed as being 'paediatric'. To avoid double counting two trials in diabetes, three in mental health, and one in neurological were classified 
within the paediatric category only. 
PPI, patient and public involvement. 
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Figure 3 and table 5 provide the data by year for spe- 
cific conditions. Both of the diabetes trials were in chil- 
dren and were coded as paediatric, and there were no 
HIV/ AIDS trials within the cohort. Figure 3 suggests 
declining rates of PPI in paediatric and mental health, 
with other areas including the general other category 
demonstrating an increase over time; however, as shown 
in table 5, the numbers in some categories were small. 

Table 6 shows the associations between trial design 
characteristics and characteristics of the condition under 
study, with consideration of PPI within the outline 
application. 

Of the intervention aims, numbers were too small to 
draw conclusions about involvement in diagnostic 
studies, but prevention trials more frequently described 
PPI than intervention trials. Trials involving educational, 
behavioural or physical interventions were more likely to 
have provided details of PPI in the outline application 



100 

90 

£ 80 

■| 70 

J 60 

re 

u 

a. 50 

Q. 
re 

o 40 
a 

HQ 

re 

£ 30 

01 

| 20 

10 




0 


2003 


2004 


2005 


2006 


2007 


2008 


% 


100 


67 


67 


45 


39 


73 



than trials involving drugs or devices. Surgical trials were 
significantly less likely to have provided details of PPI. 

The settings for trial recruitment did not appear to 
impact on specification of PPI, while recruiting partici- 
pants at the point of diagnosis was associated with less PPI. 

Forty-three trials were described as being blind and 
these trials were associated with increased frequency of 
describing PPI. Of these trials 23 involved blinding of 
the outcome assessor only. Only 14 trials involved a 
placebo, and of these, participants in 5 trials all received 
an active intervention with the placebo used as a double 
dummy. The allocation of a placebo only to one arm of 
the trial was significantly associated with greater fre- 
quency of PPI detail. 

Only two trials were in rare conditions. Both of these 
trials provided detail of PPI within the outline applica- 
tion. While there appeared to be some increase in 
describing PPI when the condition was long term there 
did not appear to be any influence based on impact of 
life expectancy. 

Board comments on outline applications 

NIHR HTA Board minutes were available for 77 (77/110 
70%) outline applications. Only nine (9/77 12%) Board 



Figure 2 Percentage of outline applications containing 
patient and public involvement (PPI) details by the year in 
which the application was made. The number of trials 
included within each year is indicated at the top of each bar. 



Table 5 Number of cumulative applications by disease/ 


condition category 








Cumulative number of applications by 


year 








2003 


2004 2005 2006 


2007 


2008 


Mental health 1 


4 4 6 


10 


13 


Cancer 


1 1 


3 


3 


Paediatrics 


5 8 


11 


15 


Degenerative 
neurological 
Pregnancy and 
childbirth 


1 


2 
1 


4 
2 


Stroke 


1 


1 


4 


Other 


3 16 


36 


49 
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minutes gave feedback on PPI representation and two of 
these did so indirectly (table 7). One comment was sup- 
portive of the PPI described in the outline, which 
involved a PPI contributor as a co-applicant. 

The corresponding outline applications were also 
available for 64 (84%) of the 77 applications for which 
Board minutes were available. Of these applications 25 
(25/64=39%) gave no information about PPI. 

Eight of the nine sets of Board comments that were 
made about PPI expressed the need for applicants to 
increase PPI. Six contained no detail about PPI within 



the application. Of these six, two were drug trials, three 
were exercise interventions and one was comparing an 
invasive with a non-invasive intervention. Two were 
recruiting participants with addiction: smoking and 
alcohol; two were recruiting elderly participants, one was 
recruiting infants, and one was recruiting participants 
with a long-term chronic debilitating condition. 

In the two outline applications which had given some 
details on PPI, comments from the Board were to 
further increase PPI. In one application the PPI contri- 
butors were employed within an NIHR condition-specific 



Table 6 PPI specified within the outline application by condition and trial characteristics 



PPI details in the outline 
application 





Ypq 

I CO 


Mn 


Tntal 




Intervention 


n (%) 

1 1 i /Of 


n (%) 


N=90 n (%) 


d Value 


Aim of intervention 










Treatment 


40 (54.1) 


34 (45.9) 


74 (82.2) 


0.66* 


Prevention 


8 (61.5) 


5 (38.5) 


41 (45.6) 




Diagnostic 


1 (33.3) 


2 (66.7) 


3 (3.3) 




Nature of interventiont 










Drug 


16 (51.6) 


15 (48.4) 


31 (34.4) 


0.70 


Device 


7 (46.7) 


8 (53.3) 


15 (16.7) 


0.51 


Surgery 


4 (25.0) 


12 (75.0) 


16 (17.8) 


0.01 


Education 


6 (60.0) 


4 (40.0) 


10 (11.1) 


0.75* 


Behavioural 


11 (64.7) 


6 (35.3) 


17 (18.9) 


0.35 


Physical 


8 (72.7) 


3 (27.3) 


11 (12.2) 


0.19 


Settingt 










Primary care 


10 (47.6) 


1 1 (52.4) 


21 (23.3) 


0.47 


Secondary care 


30 (56.6) 


23 (43.4) 


53 (58.9) 


0.62 


Emergency care 


4 (50.0) 


4 (50.0) 


8 (8.9) 


1.00* 


Community 


6 (50.0) 


6 (50.0) 


12 (13.3) 


0.77 


Social care 


4 (66.7) 


2 (33.3) 


6 (6.7) 


0.68* 


Tertiary 


4 (66.7) 


2 (33.3) 


6 (6.7) 


0.68* 


Blinding 










Yes* 


30 (69.8) 


13 (30.2) 


43 (47.8) 


0.01 


No 


19 (40.4) 


28 (59.6) 


47 (52.2) 




Involved a placebo 










No placebo 


39 (51.3) 


37 (48.7) 


76(84.4) 


0.17 


Placebo§ 


10 (71.4) 


4 (28.6) 


14 (15.6) 




Received an active intervention 










Received Placebo only]] 


8 (88.9) 


1 (11-1) 


9 (10.0) 


0.04* 


Received an active intervention 


41 (50.6) 


40 (49.4) 


81 (90.0) 




Recruitment at diagnosis 










Yes 


8 (33.3) 


16(66.7) 


24 (26.7) 


0.02 


No 


41 (62.1) 


25 (37.9) 


66 (73.3) 




Long-term condition 










Yes 


31 (60.8) 


20 (39.22) 


51 (56.7) 


0.17 


No 


17 (47.2) 


19(52.8) 


36 (40) 




Impact of condition on life expectancy 










None 


36 (57.1) 


27 (42.9) 


63 (70.0) 


0.72 


General shortening 


7 (50.0) 


7 (50.0) 


14 (15.6) 




Rapid mortality 


6 (46.2) 


7 (53.9) 


13 (14.4) 





*Fishers exact test. 

tCategories are not mutually exclusive. 
^Includes 23 trials in which only the outcome 
§Five trials used a double-dummy technique. 
HGroups receiving placebo did not receive an 
PPI, patient and public involvement. 



assessor was blind, 
active intervention. 
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Table 7 


Board comments on outline applications relating to patient and public involvement (PPI) 




Year outline 




Unique 


application 




identifier 


submitted 


Text from Board minutes for outline applications 


28 




The applicants should consider involvement of disadvantaged groups. 


65 


2006 


There was no clear service user involvement and this needs to be addressed. 


70 


^UUb 


The Board would be pleased to see letters of support from appropriate PCTS and patient 






groups that the trial is feasible. The Board wish to see patient and public involvement in any 






full proposal. 


34 


2008 


The Board noted it was a well designed study that has received input from patients. 


92 


2007 


Ethical aspects including acceptability to parents must be fully considered. 


39 


2008 


Consideration should be given to increasing service user involvement. 


42 


2008 


Patient representation is required. 


98 


2007 


An explanation and demonstration of acceptability to parents must be included. 


105 


2007 


The application would benefit from strong patient and public involvement. 





network in roles relating to PPL These individuals also 
had relevant experience as carers or patients of the con- 
dition. In the other application the PPI contributor was 
a co-applicant with an unrelated clinical background 
(midwife) and was the mother of a child with the condi- 
tion being studied. 



DISCUSSION 
Summary 

This is the first study to systematically investigate whether 
and how PPI is described within the earliest stages of trial 
development across a cohort of studies. The cohort 
covers a variety of clinical conditions and interventions, 
and this study is the first to examine associations between 
trial design, intervention and condition characteristics 
and applicants' plans for PPI. Key findings are sum- 
marised in box 2. PPI was described in only half of the 
applications at the outline stage, with only a quarter indi- 
cating that PPI contributions were enlisted at the outline 
application stage itself. There was some evidence to 
suggest that the further the trial deviates from routine 
clinical practice the more likely the application is to 
describe PPI, and PPI was particularly frequent in appli- 
cations for blinded trials or trials allocating participants 
to placebo only This may indicate the beginning of a risk- 
based approach to PPI. However, applications for trials 
recruiting at the point of diagnosis or trials with surgical 
interventions were less likely to describe PPI. For trials 
recruiting at the time of diagnosis this may reflect the dif- 
ficulty of identifying PPI representatives close to the time 
of their diagnosis, while for surgical trials this may reflect 
the lower frequency and therefore evolution of research 
practices within their RCTs. 

There were some indications that planned PPI contri- 
butions would increase as the application progressed 
from outline to full application through to funded trial. 
However, the proportion of applications describing PPI 
did not appear to increase across the years of the 
cohort. 



Strengths and weaknesses of the study 

This study used a cohort of trials funded by the NIHR 
HTA programme. To be eligible for inclusion within the 
cohort, applicants were required to have received 
funding for the trial between 2006 and 2010. 
Documentation could not be made available for unsuc- 
cessful applications. Consequently this study does not 
link details of PPI within the application to the success- 
ful award of the grant. However, while the HTA 
Programme encouraged PPI during this period, via its 
guidance notes to applicants and web information, and 
allowed it to be budgeted for, PPI was not mandatory 
within the period of the cohort, and that on its own 
would not have led to a decision that the outline appli- 
cation should not progress. As the cohort was identified 
by receipt of funding during the period rather than on 



Box 2 



What is already known on this topic 

► There is an increased awareness of the need for patient/public 
involvement in research 

► There is an expectation from funders of research that patient 
and public involvement (PPI) should be incorporated within 
that research 

► There is selective reporting of PPI within the published 
literature. 

What this study adds 

► A systematic analysis of PPI in a cohort of randomised con- 
trolled trials receiving funding from the National Institute of 
Health Research Health Technology Assessment programme 
between 2006 and 2010 

► Only half of applications consider PPI within the early stages 
of the development of the research, with only a quarter having 
involvement within the development of the outline application 
itself 

► There is evidence that particular conditions, and design con- 
siderations, impact on whether PPI is likely to be considered 
within the early stages of development 

► There is insufficient consideration of PPI at the early stages by 
funding Boards. 



8 



Gamble C, etai. BMJ Open 2014;4:e005234. doi:1 0.1 1 36/bmjopen-201 4-005234 



e 



Open Access 



the year that the outline application was submitted for 
consideration, fewer trials were available in the extremes 
of the cohort, but this approach was a consequence of 
the archiving system. The key limitation is the historical 
nature of this cohort. The period of the cohort should 
be considered when generalising the findings to the 
present day, due to revisions to the guidance for appli- 
cants and application forms in relation to PPL From 
2012 HTA boards have started to include PPI member- 
ship, and the standard NIHR application form has been 
introduced, with revised sections requiring applicants to 
clearly define their PPI involvement with guidance notes 
to applicants clearly stating that there is now an 'expect- 
ation' for 'active involvement' of patients and the public 
in the research it supports (http://www.nets.nihr.ac. 
uk/_data/assets/pdf_file/0005/67685/OutlineGuidance 
NotesSAF.pdf, last accessed 28 April 2014). In addition, 
attitudes towards PPI may have changed and this may 
influence how early researchers engage with PPI contri- 
butors. However, further research is needed to consider 
this and its potential for impact. 

It has been argued that researchers use a tokenistic 
approach to patient involvement and the level of detail 
on PPI plans within this cohort may lend some strength 
to that argument. In order to summarise the details of 
PPI across the cohort some means of categorising 
descriptions of PPI activity was necessary. The lack of 
detail provided within the applications required clear 
coding rules to be established; however, these were 
clearly documented. Establishing coding rules was con- 
sidered reasonable, as within the peer review of an 
outline application similar assumptions would need to 
be made by reviewers considering the PPI descriptions 
provided. 

The results should be interpreted with some caution, 
owing to the small numbers within some categories, and 
this caution applies to the results of the statistical tests, 
which were not powered to detect differences and 
should be considered as exploratory. 

Strengths and weaknesses in relation to other studies 

There are many examples or accounts of PPI within the 
published literature focusing on particular trials or topic 
areas 9-12 and concerns have been raised about selective 
reporting. 1 This is the first study to systematically investi- 
gate PPI early in the development of funding applica- 
tions and across a wide range of trials. A previous paper 
on PPI within clinical trial units 13 (CTUs) concluded 
that involvement appears to be growing. While the 
majority of applications within the current cohort 
included collaborations with CTUs, we found no clear 
trend of PPI activity increasing within this time. A stand- 
ard operating procedure (SOP) for PPI in CTUs has 
been recently developed; however, its focus is on PPI 
after funding has been awarded. 14 The SOP suggests 
that 1% of the research budget be allocated to PPI; 
however, other more detailed costing models are 
available. 15 



If PPI is accepted as the best practice with early 
involvement, as it provides the greatest opportunity to 
maximise its contribution, then implementation of PPI 
and its descriptions within outline applications need to 
be increased and improved with justifications and 
methods for the approaches taken. It is noteworthy that 
no application within the cohort examined provided this 
level of information. The NIHR HTA programme has 
made changes to the outline application form, which 
now requests details of PPI and justification if it is 
absent. This may lead to increased descriptions but may 
not be associated with better involvement: a view which 
has some support from the current guidance, which 
states, "Whilst patient and public involvement (PPI) may 
not always be needed for all types of research, it is always 
relevant for HTA trials. Many PPI sections on the appli- 
cation are unconvincing to our consumer referees and 
Board members." (http://www.nets.nihr.ac.uk/funding/ 
hta-researcher-led/Preparing-a-full-application-for-the- 
Clinical-Trials-and-Evaluation-Board.pdf, last accessed 
20 November 2013). 

Descriptions should specify how PPI has been incorpo- 
rated within the development of the outline application 
and the plans for involvement should the application pro- 
gress. However, the research community needs to con- 
sider the challenges facing researchers in developing PPI 
at the early design stage. In particular, the often short 
timeframes within which outline applications have to be 
developed and PPI contributors identified, and the lack 
of funding to support this work prior to the grant award, 
are major obstacles to enhancing PPI at an early stage in 
trial development. In England, a PPI bursary scheme has 
recently been launched by some of the NIHR Research 
Design Service (RDS), but is limited to those in receipt of 
advice from the RDS and has been shown to be benefi- 
cial. 16 17 There are few other alternatives to funding 
development activity at the pre-award stage. 15 

Although funding body policies support PPI, this 
support was not usually evident in the HTA Board feed- 
back to applicants at the outline stage. This may be due 
to the difficulties in assessing PPI given the lack of detail 
provided within applications, or indicate that at the 
outline stage the weight of the decision about progres- 
sion is on the perceived relevance of the question and 
design. When feedback about PPI was given this did not 
provide any guidance on how it should be addressed. 
Statements about the need to improve PPI should be 
supported with guidance on what PPI contributions 
would be appropriate within the trial being considered. 
Peer reviewers and Board members who are asked to 
comment on PPI should be supported in doing so. 
Adoption of critical appraisal guidelines may be benefi- 
cial in achieving this. 18 From 2012 NIHR HTA funding 
Boards include a PPI representative. The impact of this 
on the frequency of comments and their content is not 
yet known. 

Our findings also raise questions about what sort of 
experience qualifies (or disqualifies) individuals to be 
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PPI contributors, as identified within the comments 
from the Board. Regardless of whether they have rele- 
vant personal experience, should individuals with profes- 
sional clinical or research experience become PPI 
contributors? Does an individual's experience in a pro- 
fessional capacity somehow weaken their ability to adopt 
a patient or carer perspective, or does a professional 
background strengthen a person's ability to contribute 
to the research? Current NIHR HTA guidance states that 
"To achieve the aim of bringing fresh eyes to the work of 
the HTA programme a patient or member of the public 
should not normally be a health practitioner, manager 
or researcher" (http://www.hta.ac.uk/PPIguidance/ 
sectionl_2.shtml, last accessed 26 September 2013). 

There is also confusion about whether it is acceptable 
for individuals who have a PPI role in a professional cap- 
acity as well as relevant personal experience to provide 
PPI. Concerns have been previously raised about PPI 
representatives becoming professionalised, 19 and evi- 
dence is needed on whether this is a legitimate concern. 
This should be rationalised against the fact that PPI 
representatives are generally individuals who are selected 
by researchers based on their attributes, and their ability 
to 'represent' the wider population of interest may be 
questioned, however, this may not impact their ability to 
communicate their unique 'lay' perspective. 20 

Future research 

The lack of detail on PPI at the early stages of research 
development needs to be addressed. There is a keen 
interest in determining whether the perceived benefits 
of PPI lead to tangible benefits for research. The activ- 
ities of PPI contributors need to be tracked to provide 
insights on when and what type of involvement may be 
linked to impact. Further research tracing the evolution 
of PPI within this cohort is being undertaken. 
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